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(57) A device and method for intravascular ultra- 
sound imaging. An ultrasound signal transmitter and de- 
tector is introduced into and may be moved through a 
bodily lumen. The ultrasound signal transmitter and de- 
tector transmits ultrasonic signals and detects reflected 
ultrasound signals which contain information relating to 
the bodily lumen. A processor coupled to the ultrasound 
signal transmitter and detector is programmed to derive 
a first image or series of images and a second image or 
series of images from the detected ultrasound signals. 
The processor is also programmed to compare the sec- 
ond image or series of images to the first image or series 
of images respectively. The processor may be pro- 
grammed to stabilize the second image in relation to the 
first image and to limit drift. The processor may also be 
programmed to monitor the first and second images for 
cardiovascular periodicity, image quality, temporal 
change and vasomotion. it can also match the first se- 
ries of images and the second series of images. 
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Description 

Field Of The invention 

s [0001] The present invention relates to a device and method for enhanced image and signal processing for Intra- 
vascular Ultrasound ("IVUS"), and more specifically, to a device and method for processing IVUS image and signal 
information which will enhance the quality and utility of IVUS images. 

Backgro und Information 

[0002] IVUS images are derived from a beam of ultrasonic energy projected by apparatus such as a transducer or 
transducer array located around, along or at the lip of a catheter inserted within a blood vessel. An ultrasound beam 
from the apparatus is continuously rotated within the blood vessel forming a 360° internal cross sectional image, i.e., 
the image is formed in a transverse (X-Y) plane. Depending on the specific apparatus configuration, the image may 
is be derived either from the same transverse plane of the apparatus or from a transverse plane found slightly forward 
(i.e., distal) of the transverse plane of the apparatus. If the catheter is moved inside and along the blood vessel (I.e., 
along the Z-axis), images of various segments (series of consecutive cross sections) of the vessel may be formed and 
displayed. 

[0003] IVUS may be used in all types of blood vessels, including but not limited to arteries, veins and other peripheral 
20 vessels, and in all parts of a body. 

[0004] The ultrasonic signal that is received (detected) is originally an analog signal. This signal is processed using 
analog and digital methods so as to eventually form a set of vectors comprising digitized data. Each vector represents 
the ultrasonic response of a different angular sector of the vessel, I.e., a section of the blood vessel. The number of 
data elements in each vector (axial sampling resolution) and the number of vectors used to scan a complete cross 
25 section (lateral sampling resolution) of the vessel may vary depending on the type of system used. 

[0005] The digitized vectors may initially be placed into a two-dimensional array or matrix having Polar coordinates, 
i.e., A(r, 9). In this Polar matrix, for example, the X axis corresponds to the r coordinate and the Y axis corresponds to 
the e coordinate. Each value of the matrix is a value (ranging from 0-255 if the system is 8 bit) representing the strength 
of the ultrasonic response at that location. 
30 [0006] This Polar matrix is not usually transferred to a display because the resultant image will not be easily inter- 
preted by a physician. The information stored in the Polar matrix A(r, 6) usually undergoes several processing stages 
and is interpolated into Cartesian coordinates, e.g., X and Y coordinates (A(X, Y)) that are more easily interpreted by 
a physician. Thus, the X and Y axis of matrix A{X, Y) will correspond to the Cartesian representation of the vessel's 
cross-section. The information in the Cartesian matrix possibly undergoes further processing and is eventually dis- 
ss played for analysis by a physician. Images are acquired and displayed in a variable rate, depending on the system. 
Some systems can acquire and display images in video-display rate, e.g., up to about 30 images per second. 
[0007] IVUS examination of a segment of a bodily lumen, i.e., vessel is generally performed by situating the catheter 
distal (i.e., downstream) to the segment to be reviewed and then the catheter is pulled back (pullback) slowly along 
the bodily lumen (Z-axis) so that successive images that form the segment are continuously displayed. In many cases 
40 the catheter is connected to a mechanical pulling device which pulls the catheter at a constant speed (i.e., a typical 
speed is approximately 0.5 - 1 mm/sec). 

[0008] In IVUS imaging systems today the technique described above for displaying an image of a cross section of 
a bodily lumen, e.g., blood vessel, is generally used. These systems are deficient, however, because they do not 
include any form of stabilization of the images to compensate for movements of the catheter and/or bodily lumen, e. 

is g., blood vessel. It is well known that during IVUS imaging of a bodily lumen, there is always motion exhibited by the 
catheter and/or the bodily lumen. This motion might be exhibited in the transverse (X-Y) plane, along the vessel axis 
(Z axis) or a combination of those movements. The imaging catheter can also be tilted in relation to the vessel so that 
the imaging plane is not perpendicular to the Z axis (This movement shall be termed as angulation). These- movements 
are caused by, among other things, beating of the heart, blood and/or other fluid flow through the lumen, vasomotion, 

50 forces applied by the physician, and other forces caused by the physiology of the patient. 

[0009] In IVUSsystemstoday.whentheimagingcatheterisstationaryorwhenperformingslowmanualormechanical 
pullback, relative movement between the catheter and the lumen is the primary factor for the change in appearance 
between successive images, i.e., as seen on the display and/or on film or video. This change in appearance occurs 
because the rate of change of an image due to movements is much greater than the rate of change in the real mor- 

55 phology due to pullback. 

[0010] Stabilization occurs when the images include compensation for the relative movement between the catheter 
and the lumen in successive images. Because none of the IVUS systems used today perform stabilization, there is no 
compensation for or correction of relative movements between the catheter and the lumen. As a result, morphological 
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features are constantly moving or rotating, i.e., on the display and/or film or video. This makes it difficult for the physician 
to accurately interpret morphology in an IVUS dynamic display. Furthermore, when non-stabilized IVUS images are 
fed as an input to a processing algorithm such as 3D reconstruction or different types of filter that process a set of 
successive images, this can lead to degraded performance and misdiagnosis or inaccurate determinations, 
s [0011] Current IVUS imaging apparatus or catheters may have occasional malfunctions of an electronic or mechan- 
ical origin. This can cause displayed images to exhibit both recognized or unrecognized artifacts and obscure the real 
morphology. Currently there is no automatic methods to determine whether images posses these types of artifacts 
which hamper the analysis of the images of the vessel or bodily lumen. 

[001 2] The behavior of cardio-vascular function is generally periodic. The detection of this periodicity and the ability 
io to establish correlation between an image and the temporal phase in the cardiac cycle to which it belongs is referred 
to as cardiac gating. 

[0013] Currently, cardiac gating is performed by using an external signal, usually an ECG (Electro-Cardiogram). 
However, ECG gating requires both the acquisition of the ECG signal and its interleaving (or synchronization) with the 
IVUS image. This requires additional hardware/software. 

is [0014] Morphological features in IVUS images of blood vessels can be broken into three general categories: the 
lumen, i.e., the area through which the blood or other bodily fluid flows; the vessel layers; and the exterior, i.e., the 
tissue or morphology outside of the vessel. Blood in most IVUS films (images) is characterized by a rapidly changing 
speckular pattern. The exterior of the vessel also alternates with high temporal frequency. Currently, the temporal 
behavior of pixels and their textural attributes are not monitored automatically. 

20 [0015] Vasomotion in the context of bodily lumens, e.g., blood vessel, is defined as the change in the caliber of the 
lumen, e.g., vessel. This change can be brought about by natural circumstances or under induced conditions. Vaso- 
motion can have a dynamic component, i.e., dynamic change of the lumen's dimensions, e.g., vessel's caliber (con- 
traction and dilation) during the cardiovascular cycle, and a baseline static component, i.e., a change in the baseline 
caliber of the lumen, e.g., vessel, 

25 [0016] Vasomotion can be expressed as quantitative physiological parameters indicating the ability of the lumen, e. 
g., vessel to change its caliber under certain conditions. These types of parameters have current and possibly future 
medical and diagnostic importance in providing information regarding the state of the lumen, e.g., vessel and the effect 
of the therapy performed. 

[0017] IVUS can be used to monitor vasomotion because it provides an image of the lumen's baseline caliber and 
30 jts dynamic changes. Additionally, IVUS can be used to monitor whether the vasomotion is global (uniform), i.e., where 
the entire cross-section of the lumen contracts/dilates in the same magnitude and direction. IVUS can also be used to 
determine whether the vasomotion is non-uniform which leads to local changes in the caliber of the lumen, i.e., different 
parts of the lumen cross-section behave differently. 

[001 8] Currently, all types of vasomotion monitoring by IVUS are performed manually. This is tedious, time consum- 
es ing, and prevents monitoring of the vasomotion in real time. 

[0019] Interpretation of IVUS images is achieved through analysis of the composition of the static images and mon- 
itoring their temporal behavior. Most IVUS images can be divided into three basic parts. The most inner section is the 
flow passage of the lumen, i.e., the cavity through which matter, i.e., blood, flows. Around the flow passage is the actual 
vessel, which may include blood vessels and any other bodily vessels, which is composed of multiple layers of tissue 
w (and plaque, if diseased). Outside the vessel other tissue which may belong to the surrounding morphology, for example, 
the heart in a coronary vessel image. 

[0020] When the IVUS film is viewed dynamically, i.e., in film format, the pixels corresponding to matter flowing 
through the vessel and to the morphology exterior to the vessel exhibit a different temporal behavior than the vessel 
itself. For example, in most IVUS films, blood flowing through the vessel is characterized by a frequently alternating 
45 speckular pattern. The morphology exterior to the vessel also exhibits frequent alternation. Currently the temporal 
behavior of pixels in dynamic IVUS images is not monitored automatically. 

[0021] In current IVUS displays, if designed into the system, high frequency temporal changes are suppressed by 
means such as averaging over a number of images. However, this sometimes fails to suppress the appearance of 
features with high amplitudes, i.e., bright gray values, and it also has a blurring effect. 
so [0022] The size of the Tow passage of the lumen is a very important diagnostic parameter. When required for diag- 
nosis, it is manually determined by, for example, a physician. This is accomplished by drawing the contour of the flow 
passage borders superimposed on a static image, e.g., frozen on video or on a machine display. This method of manual 
extraction is time consuming, inaccurate and subject to bias. 

[0023] Currently, there is commercial image processing software for the automatic extraction of the flow passage. 
55 However, these are based on the gray value composition of static images and do not take into account the different 
temporal behavior exhibited by the material, e.g., blood flowing through the passage as opposed to the vessel layers. 
[0024] During treatment of vessels, it is common practice to repeat IVUS pullback examinations in the same vessel 
segments. For example, a typical situation is first to review the segment in question, evaluate the disease (if any), 
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remove the IVUS catheter, consider therapy options, perform therapy, e.g., PTCA-"balioon" or stenting, and then im- 
mediately thereafter reexamine the treated segment using IVUS in order to assess the results of the therapy. To properly 
evaluate the results and fully appreciate the effect of the therapy performed, it is desirable that the images of the pre- 
treated and post-treated segments, which reflect cross sections of the vessel lying at the same locations along the 
s vessel's Z-axis {i.e., corresponding segments), be compared. To accomplish this comparison it must be determined 
which locations in the films of the pre-treatment IVUS images and post-treatment IVUS images correspond to one 
another. This procedure, called matching (registration) allows an accurate comparison of pre- and post-treatment IVUS 
images. 

[0025] Currently, matching is usually performed by viewing the IVUS pullback films of pre- and post-treatment seg- 
10 ments, one after the other or side by side by using identifiable anatomical landmarks to locate the sequences that 
correspond visually to one another. This method is extremely imprecise and difficult to achieve considering that the 
images are unstable and often rotate and/or move around on the display due to the absence of stabilization and because 
many of the anatomical landmarks found in the IVUS pullback film of the pre-treatment segment may be disturbed or 
changed as a result of the therapy performed on the vessel. Furthermore, the orientation and appearance of the vessel 
is is likely to change as a result of a different orientations and relative positions of the IVUS catheter in relation to the 
vessel due to its removal and reinsertion after therapy is completed. The matching that is performed is manual and 
relies primarily on manual visual identification which can be extremely time consuming and inaccurate. 

Summary Of The invention 

20 

[0026] The present invention solves the problems associated with IVUS imaging systems currently on the market 
and with the prior art by providing physicians with accurate IVUS images and image sequences of the morphology 
being assessed, thereby enabling more accurate diagnosis and evaluation. 

[0027] The present invention processes IVUS image and signal information to remove distortions and inaccuracies 
25 caused by various types of motion in both the catheter and the bodily lumen. This results in both enhanced quality and 
utility of the IVUS images. An advantage provided by the present invention is that individual IVUS images are stabilized 
with respect to prior image(s), thereby removing negative effects on any later processing of multiple images. If the 
movements in each image are of the transverse type, then it is possible for the motion to be completely compensated 
for in each acquired image. 

30 [0028] The present invention also allows volume reconstruction algorithms to accurately reproduce the morphology 
since movement of the bodily lumen is stabilized. The present invention is applicable to and useful in any type of system 
where there is a- need to stabilize images (IVUS or other) because a probe (e.g., ultrasonic or other) moving through 
a lumen experiences relative motion (i.e., of the probe and/or of the lumen). 

[0029] The present invention provides for detection of an ultrasonic signal emitted by ultrasonic apparatus in a bodily 
35 lumen, conversion of the received analog signal into Polarcoordinates (A(r, 9)), stabilization in the Polar field, converting 
the stabilized Polar coordinates into Cartesian coordinates (A(X, Y)), stabilization in the Cartesian field and then trans- 
ferring the stabilized image as Cartesian coordinates to a display. Stabilized images, either in Polar or Cartesian co- 
ordinates, may be further processed prior to display or they might not be displayed. Conversion into Cartesian coor- 
dinates and/or stabilization in the Cartesian field may be done at any point either before or after stabilization in the 
40 Polar field. Additionally, either of Polar or Cartesian stabilization may be omitted, depending on the detected shift in 
the image and/or other factors. Furthermore, additional forms of stabilization may be included or omitted depending 
on the detected shift and/or other factors. 

[0030] For example, stabilization of rigid motion may be introduced to compensate for rotational motion (angular) or 
global vasomotion (expansion or contraction in the r direction) in the Polar field and/or for Cartesian displacement (X 

is and/or Y direction) in the Cartesian field. 

[0031] Transverse rigid motion between the representations of successive images is called a "shift," i.e., a uniform 
motion of all morphological features in the plane of the image. To stabilize IVUS images, the first step that is performed 
is "shift evaluation and detection." This is where the shift (if any) between each pair of successive images is evaluated 
and detected. The system may utilize a processor to perform an operation on a pair of successive IVUS images to 

50 determine whether there has been a shift between such images. The processor may utilize a single algorithm or may 
select from a number of algorithms to be used in making this determination. 

[0032] The system utilizes the algorithm(s) to simulate a shift in an image and then compares this shifted image to 
its predecessor image. The comparisons between images are known as closeness operations which may also be 
known in the prior art as matching. The system performs a single closeness operation for each shift. The results of the 
55 series of closeness operations is evaluated to determine the location (direction and magnitude) of the shifted image 
that bears the closest resemblance to the predecessor unshifted image. An image can of course be compared in the 
same manner to its successor image. After the actual shift is determined, the current image becomes the predecessor 
image, the next image becomes the current image and the above operation is repeated. 
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[0033] Using shift evaluation and detection, the system determines the type of transverse shift, e.g., rotational, ex- 
pansion, contraction, displacement (Cartesian), etc., along with the direction and magnitude of the shift. The next step 
is "shift implementation." This is where the system performs an operation or a series of operations on successive IVUS 
images to stabilize each of the images with respect to its adjacent predecessor image. This stabilization utilizes one 

5 or multiple "reverse shifts" which are aimed at canceling the detected shift. The system may include an algorithm or 
may select from a number of algorithms to be used to implement each "reverse shift." The logic which decides upon 
what reverse shift will actually be implemented on an image, prior to its feeding to further processing or display, is 
referred to as "shift logic". Once the IVUS images are stabilized for the desired types of detected motion, the system 
may then transfer the Cartesian (or Polar) image information for further processing and finally for display where the 

10 results of stabilization may be viewed, for example, by a physician. Alternatively, stabilization can be invisible to the 
user in the sense that stabilization can be used prior to some other processing steps, after which, resulted images are 
projected to the display in their original non-stabilized posture or orientation. 

[0034] It is possible that the transverse motion between images will not be rigid but rather of a local nature, i.e., 
different portions of the image will exhibit motion in different directions and magnitudes. In that case the stabilization 
fs methods described above or other types of methods can be implemented on a local basis to compensate for such 
motion. 

[0035] The present invention provides for detection of the cardiac periodicity by using the information derived only 
from IVUS images without the need for an external signal such as the ECG, This process involves closeness operations 
which are also partly used in the stabilization process. One important function of detecting periodicity {i.e.. cardiac 

20 gating), when the catheter is stationary or when performing controlled IVUS pullback, is that it allows the selection of 
images belonging to the same phase in successive cardiac cycles. Selecting images based on the cardiac gating will 
allow stabilization of all types of periodic motion (including transverse, Z-axis and angulations) in the sense that images 
are selected from the same phase in successive heart-beats. These IVUS images, for example, can be displayed and 
any gaps created between them may be compensated for by filling in and displaying interpolated Images. The IVUS 

25 images selected by this operation can also be sent onward for further processing. 

[0036] The closeness operations used for periodicity detection can also be utilized for monitoring image quality and 
indicate artifacts associated with malfunction of the imaging and processing apparatus. 

[0037] Operations used for shift evaluation can automatically indicate vasomotion. This can serve the stabilization 
process as vasomotion causes successive images to differ because of change in the vessel's caliber. If images are 

so stabilized for vasomotion, then this change is compensated for. Alternatively, the information regarding the change in 
caliber may be displayed since it might have physiological significance. Monitoring of vasomotion is accomplished by 
applying closeness operations to successive images using their Polar representations, i.e., A(r, 6). These operations 
can be applied between whole images or between corresponding individual Polar vectors (from successive images), 
depending on the type of information desired. Since global vasomotion is expressed as a uniform change in the lumen's 

35 caliber it can be assessed by a closeness operation which takes into account the whole Polar image. In general, any 
operation suitable for global stabilization in the Polar representation can be used to assess global vasomotion, 
[0038] Under certain conditions during IVUS imaging there may be non-uniform vasomotion, i.e., movement only in 
certain sections of the IVUS image corresponding to specific locations in the bodily lumen. This may occur, for example, 
where an artery has a buildup of plaque in a certain location, thereby allowing expansion or contraction of the artery 

to only in areas free of the plaque buildup. When such movement is detected the system is able to divide the ultrasound 
signals representing cross sections of the bodily lumen into multiple segments which are then each processed individ- 
ually with respect to a corresponding segment in the adjacent image using certain algorithm(s). The resulting IVUS 
images may then be displayed. This form of stabilization may be used individually or in conjunction with the previously 
discussed stabilization techniques. Alternatively, the information regarding the local change in vessel caliber can be 

is displayed since it might have physiological significance. 

[0039] The temporal behavior of pixels and their textural attributes could serve for: enhancement of display; and 
automatic segmentation (lumen extraction). If monitored in a stabilized image environment then the performance of 
the display enhancement and segmentation processes may be improved. 

[0040] According to the present invention, the temporal behavior of IVUS images may be automatically monitored. 
so The information extracted by such monitoring can be used to improve the accuracy of IVUS image interpretation. By 
filtering and suppressing the fast changing features such as the matter, e.g., blood flowing through the vessel and the 
morphology exterior to the vessel as a result of their temporal behavior, human perception of the vessel on both static 
images and dynamic images, e.g., images played in cine form, may be enhanced. 

[0041] Automatic segmentation, i.e., identification of the vessel and the matter, e.g., blood flowing through the vessel 
55 may be performed by using an algorithm which automatically identifies the matter, e.g., blood based on the temporal 
behavior of textural attributes formed by its comprising pixels. The temporal behavior that is extracted from the images 
can be used for several purposes. For example, temporal filtering may be performed for image enhancement, and 
detection of the changes in pixel texture may be used for automatic identification of the lumen and its circumference. 
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[0042] In all IVUS images, the catheter itself (and imaging apparatus) is best to be eliminated from the image prior 
to performing stabilization or for monitoring. Failure to eliminate the catheter might impair stabilization techniques and 
monitoring. Elimination of the catheter may be performed automatically since its dimensions are known. 
[0043] The present invention also provides for automatic identification {i.e., matching or registration) of corresponding 
frames of two different IVUS pullback films of the same segment of a vessel, e.g., pre-treatment and post-treatment. 
To compare a first IVUS pullback film, i.e., a first IVUS imaging sequence, with a second IVUS pullback film, i.e., a 
second IVUS imaging sequence, of the same segment of a bodily lumen, for example, captured on video, film or in 
digitized form, the imaging sequences must be synchronized. Matching, which will achieve this synchronization, in- 
volves performing closeness operations between groups of consecutive images belonging to the two sets of IVUS 
imaging sequences. 

[0044] Out of one imaging sequence a group of consecutive images, termed the reference group, is selected. This 
group should be selected from a portion of the vessel displayed in both imaging sequences and it should be a portion 
on which therapy will not be performed since the morphology of the vessel is likely to change due to therapy. Another 
condition for this matching process is that the two imaging sequences are acquired at a known, constant and preferably 
the same pullback rate. 

[0045] Closeness operations are performed between the images of the reference group and the images from the 
second group which has the same number of successive images extracted from the second imaging sequence. This 
second group of images is then shifted by a single frame with respect to the reference group and the closeness oper- 
ations are repeated. This may be repeated for a predetermined number of times and the closeness results of each 
frame shift are compared to determine maximal closeness. Maximal closeness will determine the frame displacement 
between the images of the two imaging sequences. This displacement can be reversed in the first or second film so 
that corresponding images may be automatically identified and/or viewed simultaneously. 

[0046] Thus, corresponding images may be viewed, for example, to determine the effectiveness of any therapy 
performed or a change in the morphology over time. Additionally, the various types of stabilization discussed above 
may be implemented within or between the images in the two sequences, either before, during or after this matching 
operation. Thus, the two films can be displayed not only in a synchronized fashion, but also in the same orientation 
and posture with respect to one another. 

Brief Description Of The Drawings 

[0047] 

Figures 1 (a) and (b) show a two-dimensional array or matrix of an image arranged in digitized vectors in Polar and 
Cartesian coordinates, respectively. 

Figure 2 illustrates the results of a shift evaluation between two successive images in Cartesian coordinates. 

Figure 3 shows images illustrating the occurrence of drift phenomena in Polar and Cartesian coordinates. 

Figure 4 illustrates the effect of performing stabilization operations (rotational and Cartesian shifts) on an image. 

Figure 5 illustrates global contraction or dilation of a bodily lumen expressed in the Polar representation of the 
image and in the Cartesian representation of the image. 

Figure 6 shows an image divided into four sections for processing according to the present invention. 

Figure 7 shows a vessel, in both Cartesian and Polar coordinates, in which local vasomotion has been detected. 

Figure 8 illustrates the results of local vasomotion monitoring in a real coronary vessel In graphical form. 

Figure 9 shows an ECG and cross-correlation coefficient plotted graphically in synchronous fashion. 

Figure 10 shows a table of a group of cross-correlation coefficient values (middle row) belonging to successive 
images (numbers 1 through 10 shown in the top row) and the results of internal cross-correlations (bottom row). 

Figure 11 shows a plot of a cross-correlation coefficient indicating an artifact in IVUS images. 

Figure 12 shows an IVUS images divided into three basic parts: the lumen through which fluid flows; the actual 
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vessel; and the surrounding tissue. 

Figure 13 illustrates the results of temporal filtering. 

5 Figure 14 shows an image of the resufts of the algorithm for automatic extraction of the lumen. 

Figure 15 illustrates the time sequence of a first film (left column), reference segment from the second film (middle 
column) and the images from the first film which correspond (or match) the images of the reference segment (right 
column). 

10 

Detailed Description 

[0048] In intravascular ultrasound (IVUS) imaging systems the ultrasonic signals are emitted and received by the 
ultrasonic apparatus, for example, a transducer or transducer array, processed and eventually arranged as vectors 

is comprising digitized data. Each vector represents the ultrasonic response of a different angular sector of the bodily 
lumen. The number of data elements in each vector (axial sampling resolution) and the number of vectors used to scan 
the complete cross-section (lateral sampling resolution) of the bodily lumen depends on the specific IVUS system used. 
[0049] The digitized vectors are initially packed into a two-dimensional array or matrix which is illustrated in Figure 
1(a). Generally, this matrix has what are known as Polar coordinates, i.e., coordinates A(r, e). The X-axis of the matrix 

20 shown in Figure 1(a) corresponds to the r coordinate while the Y-axis of the matrix corresponds to the r coordinate 
while the Y-axis of the matrix corresponds to the 6 coordinate. Each value of the matrix is generally a gray value, for 
example, ranging from 0-255 if it is 8 bit, representing the strength of the ultrasonic signal at that corresponding location 
in the bodily lumen. This Polar matrix may then be converted into a Cartesian matrix as shown in Figure 1{b) having 
an X-axis and Y-axis which correspond to the Cartesian representation of the vessel's cross-section. This image may 

25 then be further processed and transferred to a display. The initial array and the display may each utilize either Polar 
or Cartesian coordinates. The values for the matrix may be other than gray values, for example, they may be color 
values or other values and may be less than or more than 8 bits. 

[0050] During an IVUS imaging pullback procedure the bodily lumen, hereinafter referred to as a vessel, and/or the 
imaging catheter may experience several modes of relative motion. These types of motion include: (1) Rotation in the 

30 plane of the image, i.e., a shift in the G coordinate of the Polar image; (2) Cartesian displacement, i.e., a shift in the X 
and/or Y coordinate in the Cartesian image; (3) Global vasomotion, characterized by a radial contraction and expansion 
of the entire vessel, i.e., a uniform shift in the r-coordinate of the Polar image; (4) Local vasomotion, characterized by 
a radial contraction and expansion of different parts of the vessel with different magnitudes and directions, i.e., local 
shifts in the r-coordinate of the Polar image; (5) Local motion, characterized by different tissue motion which vary 

35 depending on the exact location within the image; and (6) Through plane motion, i.e., movements which are perpen- 
dicular or near perpendicular (angulation) to the plane of the image, 

[0051] Stabilization of successive raw images is applicable to the first 5 types of motion described above because 
motion is confined to the transverse plane. These types of motion can be compensated for, and stabilization achieved, 
by transforming each current image so that its resemblance to its predecessor image is maximized. The first 3 types 

40 of motion can be stabilized using closeness operations which compare whole or large parts of the images one to 
another. This is because the motion is global or rigid in its nature. The 4th and 5th types of motion are stabilized by 
applying closeness operations on a localized basis because different parts of the image exhibit different motion. The 
6th type of motion can be only partly stabilized by applying closeness operations on a localized basis. This is because 
the motion is not confined to the transverse plane. This type of motion can be stabilized using cardiovascular periodicity 

45 detection. 

[0052] The next sections shall describe methods for global stabilization, followed by a description of methods for 
local stabilization. Stabilization using cardiovascular periodicity detection shall be described in the sections discussing 
periodicity detection. 

[0053] To achieve global stabilization, shift evaluation is performed using some type of closeness operation. The 
50 closeness operation measures the similarity between two images. Shift evaluation is accomplished by transforming a 
first image and measuring its closeness, i.e., similarity, to its predecessor second image. The transformation may be 
accomplished, for example, by shifting the entire first image along an axis or a combination of axes (X and/or Y in 
Cartesian coordinates or r and/or 9 in Polar coordinates) by a single pixel (or more). Once the transformation, i.e., shift 
is completed the transformed first image is compared to the predecessor second image using a predefined function, 
55 This transformation is repeated, each time by shifting the first image an additional pixel (or more) along the same and/ 
or other axis and comparing the transformed first image to the predecessor second image using a predefined function. 
After all of the shifts are evaluated, the location of the global extremum of the comparisons using the predefined function 
will indicate the direction and magnitude of the movement between the first image and its predecessor second image. 
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[0054] For example, Figure 2 illustrates the results of a shift evaluation between two successive images in Cartesian 
coordinates, image A is a predecessor image showing a pattern, e.g., a cross-section of a vessel, whose center is 
situated in the bottom right quadrant of the matrix. Image B is a current image showing the same pattern but moved 
in an upward and left direction and situated in the upper left quadrant of the matrix. The magnitude and direction of 

5 the movement of the vessel's center is indicated by the arrow. The bottom matrix is the C(shiftX, shiftY) matrix which 
is the resulting matrix after performing shift evaluations using some type of closeness operation. 
[0055] There are many different algorithms or mathematical functions that can be used to perform the closeness 
operations. One of these is cross-correlation, possibly using Fourier transform. This is where the current and prede- 
cessor images each consisting of, for example, 256 x 256 pixels, are each Fourier transformed using the FFT algorithm. 

10 The conjugate of the FFT of the current image is multiplied with the FFT of the predecessor image. The result is 
inversely Fourier transformed using the I FFT algorithm. The formula for cross-correlation using Fourier transform can 
be shown as follows: 



C = real(ifft2((fft2(A)) * conj(fft2(B)))) 







A = 


predecessor image matrix (e.g., 256 x 256); 


20 B = 


current image matrix (e.g., 256 x 256); 


fft2 = 


two dimensional FFT; 


iff!2 = 


two dimensional inverse FFT; 


conj = 


conjugate; 


real = 


the real part of the complex expression; 




multiplication of element by element; and 


C = 


cross-correlation matrix. 


[0056] 


Evaluating closeness using cross-correlation implemented by Fourier transform is actually an approximation. 



This is because the mathematical formula for the Fourier transform relates to infinite or periodic functions or matrices, 
30 while in real life the matrices (or images) are of a finite size and not necessarily periodic. When implementing cross- 
correlation using FFT, the method assumes periodicity in both axes. 

[0057] As a result, this formula is a good approximation and it reflects the actual situation in the 9-axis of the Polar 
representation of the image, however, it does not reflect the actual situation in the r-axis of the Polar representation or 
of the X- or Y-axis of the Cartesian representation of the image. 
35 [0058] There are a number of advantages to cross-correlation utilizing FFT. First, all values of the cross-correlation 
matrix C(shiftX, shiftY) are calculated by this basic operation. Furthermore, there is dedicated hardware for the efficient 
implementation of the FFT operation, i.e. Fourier transform chips or DSP boards. 

[0059] Another algorithm that can be used to perform closeness operations is direct cross-correlation, either nor- 
malized or not. This is achieved by multiplying each pixel in the current shifted image by its corresponding pixel in the 
io predecessor image and summing up all of the results and normalizing in the case of normalized cross-correlation. 
Each shift results in a sum and the actual shift will be indicated by the largest sum out of the evaluated shifts. The 
formula for cross-correlation can be shown by the following formula: 



Y,8(x - shifiX, yshiftY) ■ A(x. y) 

C(shiftX, shiftY)- *' r 



so [0060] The formula for normalized cross correlation is 



C(shiftX. shiftY) = Z B(x-shiftX, y-shi/t Y)*A(x.y)l 

x.y 

vX (B(x~shiftX, y-shiftr) * B(x-shiftX, y-shiftY)) V£ (A(x. y) * A(x, y)) 
*■> x.y 
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A = predecessor image matrix; 

B = current image matrix; 

* = multiplication of pixel by corresponding pixel; 

X = sum of all pixels in matrix; 

C = matrix holding results for all performed shifts. 

[0061] Using this direct method of cross-correlation, C(shiftX, shiftY) can be evaluated for all possible values of shiftX 
and shiftY. For example, if the original matrices, A and B, have 256 x 256 pixels each, then shiftX and shiftY values, 
each ranging from -127 to + 128 would have to be evaluated, making a total of 256 x 256 = 65,536 shift evaluations 
in order for C(shiftX, shiftY) to be calculated for all possible values of shiftX and shiftY. Upon completion of these 
evaluations the global maximum of the matrix is determined. 

£0062] Direct cross-correlation can be implemented more efficiently by lowering the number of required arithmetic 
operations. In order to detect the actual shift between images, evaluation of every possible shiftX and shiftY is not 
necessary. It is sufficient to find the location of the largest C(shiftX, shiftY) of all possible shiftX and shiftY. 
[0063] A third algorithm that can be used to perform closeness operations is the sum of absolute differences (SAD). 
This is achieved by subtracting each pixel in one image from its corresponding pixel in the other image, taking their 
absolute values and summing up all of the results. Each shift will result in a sum and the actual shift will be indicated 
by the lowest sum. The formula for sum of absolute differences (SAD) can be shown as follows: 

SAD = absolute(A - B) 

[0064] This formula can also be shown as follows: 



A = predecessor image matrix; 

B = current image matrix; 

abs = absolute value. 

- = subtraction of element by element; and 

£= sum of all differences. 

[0065] While the accuracy of each of these algorithms/formulas may vary slightly depending on the specific type of 
motion encountered and system settings, it is to be understood that no single formula can, a-priori be classified as 
providing the best or most accurate results. Additionally, there are numerous variations on the formulas described 
above and other algorithms/formulas that may be utilized for performing shift evaluation and which may be substituted 
for the algorithms/formulas described above. These algorithms/formulas also include those operations known in the 
prior art for use as matching operations 

[0066] Referring again to Figure 2, if the closeness operation performed is cross-correlation, then C(shiftX, shiftY) 
is called the cross-correlation matrix and its global maximum (indicated by the black dot in the upper left quadrant) will 
be located at a distance and direction from the center of the matrix (arrow in matrix C) which is the same as that of the 
center of the vessel in Image B relative to the center of the vessel in image A (arrow in Image B). 
[0067] If the closeness operation performed is SAD, then the black dot would indicate the global minimum which will 
be located at a distance and direction from the center of the matrix (arrow in matrix C) which is the same as that of the 
center of the vessel in Image B relative to the center of the vessel in Image A (arrow in Image B). 
[0068] Rotational motion is expressed as a shift along the current Polar image in the 0-coordinate relative to its 
predecessor. The rotational shift in a current image is detected by maximizing the closeness between the current Polar 
image and its predecessor. Maximum closeness will be obtained when the current image is reversibly shifted by the 
exact magnitude of the actual shift. In for example, a 256 x 256 pixel image, the value of the difference (in pixels) 
between 128 and the 0-coordinate of the maximum in the cross-correlation image (minimum in the SAD image), will 
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indicate the direction (positive or negative) and the magnitude of the rotation. 

[0069] Global vasomotion is characterized by expansion and contraction of the entire cross section of the vessel. In 
the Polar image this type of motion is expressed as movement inwards and outwards of the vessel along the r-axis. 
Vasomotion can be compensated by performing the opposite vasomotion action on a current Polar image in relation 
s to its predecessor Polar image using one of the formulas discussed above or some other formula. In contrast to angular 
stabilization, vasomotion stabilization does not change the orientation of the image but actually transforms the image 
by stretching or compressing it. 

[0070] Cartesian displacement is expressed as a shift in the X-axis and/or Y-axis in the Cartesian image relative to 
its predecessor. This type of motion is eliminated by shifting the Cartesian image in an opposite direction to the actual 
10 shift. Thus, Cartesian displacement, in the Cartesian representation, can be achieved by essentially the same arithmetic 
operations used for rotational and vasomotion stabilization in the Polar representation. 

[0071] The number of shift evaluations necessary to locate the global extremum (maximum or minimum, depending 
on the closeness function) of CfshiftX, shiftY) may be reduced using various computational techniques. One technique, 
for example, takes advantage of the fact that motion between successive IVUS images is, in general, relatively low in 

15 relation to the full dimensions of the Polar and/or Cartesian matrices. This means that C(shiftX, shiftY) can be evaluated 
only in a relatively small portion around the center of the matrix, i.e., around shiftX = 0, shiftY = 0. The extremum of 
that portion is assured to be the global extremum of matrix CfshiftX, shiftY) including for larger values of shiftX and 
shiftY. The size of the minimal portion which will assure that the extremum detected within it is indeed a global extremum 
varies depending on the system settings. The number of necessary evaluation operations may be further reduced by 

2" relying on the smoothness and monotonous property expected from the C matrix (especially in the neighborhood of 
the gioba! extremum). Therefore, if the value in the CfshiftX, shiftY) matrix at a certain location is a local extremum (e. 
g., in a 5 x 5 pixel neighborhood), then it is probably the global extremum of all of matrix C(shiftX, shiftY). 
[0072] Implementing this reduction of the number of necessary evaluations can be accomplished by first searching 
from the center of the matrix (shiftX = 0, shiftY = 0) and checking a small neighborhood, e.g., 5x5 pixels around the 

25 center. If the local extremum is found inside this neighborhood then it is probably the global extremum of the whole 
matrix CfshiftX, shiftY) and the search may be terminated. If, however, the local extremum is found on the edges of 
this neighborhood, e.g., shiftX = -2, shiftX = 2, shiftY = -2 or shiftY = 2, then the search is repeated around this pixel 
until a C(shiftX, shiftY) value is found that is bigger (smaller) than all of its close neighbors. 

[0073] Because in a large number of images there is no inter-image motion, the number of evaluations needed to 
so locate the global extremum in those cases, will be approximately 5 x 5 = 25, instead of the original 65,536 evaluations. 
[0074] The number of necessary evaluation operations may also be reduced by sampling the images. For example, 
if 256 x 256 sized images are sampled for every second pixel then they are reduced to 128 x 128 sized matrixes. In 
this case, direct cross-correlation or SAD, between such matrixes involve 128 x 128 operations instead of 256 x 256 
operations, each time the images are shifted one in relation to the other. Sampling, as a reduction method for shift 
35 evaluation operations can be interleaved with other above described methods for reduction. 

[0075] Referring again to Figure 2, as a result of the closeness operation, the indicated shiftX will have a positive 
value and shiftY a negative value. In order to stabilize Image B,/.e., compensate for the shifts in the X and Y directions, 
shift logic will reverse the shifts, i.e., change their sign but not their magnitude, and implement these shifts on the matrix 
corresponding to Image B. This will artificially reverse the shift in Image B and cause Image B to be unshifted with 
to respect to Image A. 

[0076] The actual values used in the closeness calculations need not necessarily be the original values of the matrix 
as supplied by the imaging system. For example, improved results may be achieved when the original values are raised 
to the power of 2, 3 or 4 or processed by some other method. 

[0077] The imaging catheter and the enclosing sheath appear as constant artifacts in all IVUS images. This feature 
45 obscures closeness operations performed between images since it is not part of the morphology of the vessel. It is, 
therefore, necessary to eliminate the catheter and associated objects from each image prior to performing closeness 
operations, i.e., its pixels are assigned a value of zero. The elimination of these objects from the image may be per- 
formed automatically since the catheter's dimensions are known. 

[0078] Shift evaluation and Implementation may be modular. Thus, shift evaluation and implementation may be !im- 
50 ited to either Polar coordinates or Cartesian coordinates individually, or shift evaluation and implementation may be 
implemented sequentially for Polar and Cartesian coordinates. Presently, because imaging in IVUS systems is gener- 
ally organized by first utilizing Polar coordinates and then converting into Cartesian coordinates, it is most convenient 
to perform shift evaluation and implementation in the same sequence. However, the sequence may be modified or 
changed without any negative effects or results. 
55 [0079] The shift evaluation process can be performed along one or two axis. In general, two dimensional shift eval- 
uation is preferred even when motion is directed along one axis. Shift implementation may be limited to both axis, one 
axis or neither axis. 

[0080] There is not a necessary identity between the area in the image used for shift evaluation and between the 
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area on which shift implementation is performed. For example, shift evaluation may be performed using a relatively 
small area in the image while shift implementation will shift the whole image according to the shift indicated by this area. 
[0081] A trivial shift logic is one in which the shift implemented on each image (thereby forming a stabilized image) 
has a magnitude equal, and in opposite direction, to the evaluated shift. However, such logic can result in a process 

s defined as Drift. Drift is a process in which implemented shifts accumulate and produce a growing shift whose dimen- 
sions are significant in relation to the entire image or display. Drift may be a result of inaccurate shift evaluation or non- 
transverse inter-image motion at some part of the cardiovascular cycle. When Cartesian stabilization is implemented, 
drift can cause, for example, the shifting of a relatively large part of the image out of the display. When rotational 
stabilization is implemented, drift can cause the increasing rotation of the image in a certain direction. 

10 [0082] Figure 3 is an image illustrating the occurrence of drift in Polar and Cartesian coordinates. The left image is 
the original display of the image while the right image is the same image after Polar and Cartesian stabilization has 
been performed. Note how the right image is rotated counter-clockwise in a large angle and shifted downward in relation 
to the left image. In this case, rotational and Cartesian shift implementation do not compensate for actual shifts in the 
image, but rather arise from inaccurate shift evaluation. 

is [0083] The shift logic must be able to deal with this drift so that there will be a minimal implementation of mistaken 
evaluated shifts. One method for preventing, or at least limiting drift is by setting a limit to the magnitude of allowable 
shifts. This will minimize the drift but at the cost of not compensating for some actual shift. Additional methods can be 
used to prevent or minimize shift. These may possibly be interleaved with cardio-vascular periodicity detection methods 
discussed later. 

2Q [0084] The images shown in Figure 4 illustrate the effect of performing stabilization operations (rotational and Car- 
tesian shifts) on an image. The left image is an IVUS image from a coronary artery as it would look on a large portion 
of a regular display (with catheter deleted) while the right image shows how the left image would be displayed after 
stabilization operations are implemented. 

[0085] Taking a close look at the left and right images in Figure 4, certain differences can be observed. First, the 
25 right image is slightly rotated in a clockwise direction {i.e., by a few degrees) in relation to the left image. This is the 
result of rotational stabilization. Next, the right image is translated in a general left direction in relation to the left image. 
This can be detected by noting the distance of the lumen (cavity) from the edges of the picture in each image. This is 
a result of Cartesian shift stabilization operations. 

[0086] The advantages of stabilization of the displayed image cannot be appreciated by viewing single images as 
30 shown in Figure 4. However, viewing a film of such images would readily illustrate the advantages. In a display which 
does not include stabilization, the location of the catheter would always be situated in the center of the display and the 
morphological features would move around and rotate on the display. In contrast, in a stabilized display, the location 
of the catheter would move around while the morphological features would remain basically stationary. Stabilization 
does not necessarily have to be exhibited on an actual display. It can be invisible to the user in the sense that stabilization 
35 will enhance subsequent processing steps, but the actual display will exhibit the resultant processed images in their 
original (non-stabilized) posture and orientation. 

[0087] Figure 5 illustrates global contraction or dilation of a vessel, expressed in the Polar representation of the 
image as a movement of the features along the r-coordinates, i.e., movement along the Polar vectors. Figure 5 also 
shows the same global contraction or dilation expressed in the Cartesian representation of the image. Figure 5(a) 
■to shows the baseline appearance of the cross section of a vessel in both the Polar and Cartesian representations. Figure 
5(b) shows a relative to baseline contraction of the vessel. Figure 5(c) shows a relative to baseline uniform dilation of 
the vessel. 

[0088] Since global vasomotion is expressed as a uniform change in the vessel's caliber, any operation suitable for 
stabilization in the Polar representation can be used to assess global vasomotion, e.g., it can be assessed by a close- 

45 ness operation utilizing the entire Polar image. 

[0089] Aftertwo dimensional shift evaluation is performed, as discussed above, the location of the maximum in matrix 
C(shiftX, shiftY) on the 0-axis is utilized for rotational stabilization. This leaves the location of the extremum on the r- 
axis, which can be used as an indication of global vasomotion, Thus, global vasomotion monitoring is a by-product of 
two dimensional shift evaluation in the Polar image. 

so [0090] Each pair of successive images produce a value indicative of the vasomotion. Both the magnitude and the 
sign of the resulting shift between images characterize the change in the vessel, i.e., vasomotion. Negative shifts 
indicate dilation, and positive shifts indicate contraction. The magnitude of the value indicates the magnitude of the 
vasomotion change. 

: [0091] Under certain circumstances motion or vasomotion may not be uniform/rigid although confined to the plane 
55 of the image, i.e., transverse. To determine the type of motion or vasomotion, the image maybe divided into sections 
and global stabilization evaluation performed on each of these sections. By examining the indicated shifts of these 
sections relative to the corresponding sections in the predecessor image, a determination can be made as to the type 
of motion. For example, as shown in Figure 6, the image in Figure 6(a) can be divided into four sections as shown in 
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Figure 6(b). Shift evaluation can be performed separately on each of the four sections. Comparison between the results 
of the shift evaluation for each of the four sections can possibly identify the type of actual motion. Thus, the type of 
stabilization applied can be varied depending on the type of motion detected. 

[0092] Stabilization for local motion is achieved by performing closeness operations on a localized basis. Small 
s portions of the predecessor image A ("template" regions) and small portions of the current image B ("search" regions) 
participate in the local stabilization process. Sometimes, it is best to perform local stabilization after global stabilization 
has been performed. 

[0093] During local stabilization, template regions in the predecessor image (A) are shifted within search regions 
and compared, using closeness operations to template sized regions in the current image (B). Each pixel, in the (newly) 
io. formed stabilized image (B 1 ) will be assigned a new value based on the results of the search and closeness evaluation 
performed. 

[0094] Local stabilization is illustrated by the following example in which the template region Is a 1 x 1 pixel region, 
i.e., a single pixel, the search region is a 3 x 3 pixel region and the closeness operation is SAD. In the following diagram, 
the pixel valued 3 in A and the pixel valued 9 in B are corresponding pixels. The 3 x 3 pixel neighborhood of the pixel 
is valued 9 is also illustrated. 



Pixel in A ("template" region) Pixels in B (3 x 3 "search" region) B' 

20 

I 10 10 

3 7 9 50 I 

II 7 60 

25 

[0095] In this example, according to the conditions described above the 'template' pixel valued 3 is compared using 
SAD to all pixels found in the 3 x 3 search region around the pixel valued 9. The pixel valued 1 at the top left corner 
of the search region will achieve the minimal SAD value (|1-3| = 2) out of all the possibilities in the search region. As 
a result, in the newly formed stabilized image (B'), the pixel corresponding in location to pixels valued 3 and 9 will be 

so assigned the value of 1 . 

[0096] In general, the dimensions of the template and search region can be varied along with the closeness opera- 
tions used. The actual value which is assigned to the pixel of the newly formed stabilized image (B") need not necessarily 
be an actual pixel value from the current image B (as illustrated in the example) but some function of pixel values. It 
is important to note that as a result of local stabilization, as opposed to the global/rigid methods, the "composition" of 

35 the image, i.e., the internal relationship between pixels, and their distribution in the stabilized image, changes in relation 
to the original Image, Local stabilization can be implemented on both the Polar and Cartesian representations of the 
image. 

[0097] Figure 7 shows a vessel, in both Cartesian and Polar coordinates, in which local vasomotion has been de- 
tected. When local vasomotion is detected, it is an indication that some parts of the cross-section of the vessel are 

"0 behaving differently than other parts of the cross-section. 

[0098] Figure 7(a) shows a baseline figure of the vessel prior to local vasomotion. Figure 7(b) shows an example of 
local vasomotion. As indicated in both the Cartesian and Polar representations, four distinct parts of the vessel behave 
differently: two segments of the vessel do not change caliber, or do not move relative to their corresponding segments 
in the predecessor image; one segment contracts, or moves up; and one segment dilates, or moves down. 

45 [0099] As can be observed, global vasomotion evaluation methods are not appropriate for evaluating local vasomo- 
tion because the vessel does not behave in a uniform manner, if global vasomotion evaluation was to be applied, for 
example, on the example shown in Figure 7, it might detect overall zero vasomotion, i.e. the contraction and dilation 
would cancel each other, 

[0100] Therefore, local vasomotion evaluation methods must be utilized. This may be achieved by separately eval- 
eo uating vasomotion in each Polar vector, i.e., in each 6 (or Y) vector. Closeness operations are applied using one 
dimensional shifts in corresponding Polar vectors. For example, if closeness is utilized with cross-correlation, then the 
following operation illustrates how this is accomplished using one dimensional shifts. 



C(shiftX, Y) = l B(x-shiftX, y) * A(x y) 
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where: 

A = predecessor image matrix; 
B = current image matrix; 
5 * = multiplication of pixel by corresponding pixel; 

S = sum of pixels in the matrix of the Polar vector; 
C = two dimensional matrix of correlation coefficient. 

[0101] As can be seen, shifting is performed along one axis (X or r-axis) for each and every Polar vector (8 or Y 
10 vector). The values assigned in each vector for shift evaluation may not be the actual values of the images but, for 
example, each pixel in the vector can be assigned the average of its lateral neighbors, i.e., A{X, Y) will be assigned, 
for example, the average of A(X, Y-1 ), A(X, Y) and A(X, Y+1 ). The same goes for B(shiftX, Y). This can make the cross- 
correlation process more robust to noise. 

[0102] A two dimensional matrix (C(shiftX, Y)) is formed. Each column in the matrix stores the results of closeness/ 
15 similarity operations performed between corresponding Polar vectors from the current image and the predecessor 
image. This operation could also have been implemented using FFT. 

[0103] After formation of the matrix, the location of the extremum (maximum in the cross-correlation operation) in 
each column is detected. This extremum location indicates the match between the current Polar vector and its pred- 
ecessor. Thus, the vasomotion in each vector can be characterized, i.e., the radial movement in each specific angular 
20 sector of the vessel. 

[01 04] This information can be used to display the local vasomotion, it can be added up from some orall Polar vectors 
and averaged to determine an average value for the vasomotion, or it can be used for other purposes. Therefore, by 
evaluating local vasomotion, both local and global vasomotion can be evaluated. 

[0105] To be effectively used and/or expressed as quantitative physiological parameters, the magnitude of vasomo- 
25 tion must relate in some fashion to the vessel's actual caliber. Thus, measurements of vasomotion monitoring should 
generally be used in conjunction with automatic or manual measurements of the vessel's caliber Besides for true 
vasomotion, Cartesian displacement may also be detected as vasomotion. This is because Cartesian displacement, 
when expressed in Polar coordinates, results in shifts along both the r and 9 axes. To distinguish true vasomotion from 
Cartesian displacement, shift evaluation in the Cartesian image must indicate no, or little motion. If Cartesian displace- 
so ment is detected, then it must first be stabilized. Thereafter, the Cartesian coordinates may be converted back into 
Polar coordinates for vasomotion evaluation. This will allow greater success and provide more accurate results when 
determining actual vasomotion. 

[0106] The graphs in Figure 8 illustrate the results of local vasomotion monitoring in a human coronary vessel in 
vivo. Local vasomotion monitoring was performed twice in approximately the same segment of the vessel, and consisted 

35 of 190 successive images as shown (X-axis) in Figures 8(a) and 8(b). The difference between the two graphs is that 
the vasomotion evaluation shown In Figure 8(a) was performed prior to treatment of the artery, i.e., pre-interventlon, 
while the vasomotion evaluation shown in Figure 8(b) was performed after treatment of the artery, I.e., post-intervention. 
[01 07] In every image, vasomotion was assessed locally in every Polar vector and then all detected individual shifts 
were added and averaged to produce a single global vasomotion indication (Y-axis) for each image, i.e., an indication 

*o for vasomotion activity. 

[01 08] The units on the Y-axis do not have a direct physiological meaning because the actual caliber of the vessel 
was not calculated, but the relationship between the values in Figures 8(a) and 8(b) have a meaning because they 
were extracted from the same vessel. Thus, important information may be derived from these figures. Note how the 
vasomotion increased after treatment (maximal vasomotion from approximately 40 to approximately 1 50). Therefore, 

45 even though vasomotion was not fully quantified, a change in physiology (probably linked to the treatment) has been 
demonstrated. 

[0109] Cardiovascular periodicity may be monitored solely based on information stored in IVUS images, thereby 
eliminating the need for an ECG or any other external signal. This means that a link can be established between every 
image and its respective temporal phase in the cardiovascular cycle without need for an external signal. Once this 

so linkage is established, then monitoring can substitute the ECG signal in a large number of utilities which require cardiac 
gating. This monitoring may be accomplished using closeness operations between successive images. Moreover, the 
same closeness operations can produce information regarding the quality of IVUS images and their behavior. 
[0110] The cardiac cycle manifests itself in the cyclic behavior of certain parameters that are extracted by IVUS 
images. If the behavior of these parameters are monitored, then the periodicity of the cardiac cycle can be determined. 

55 Knowing the frame acquisition rate will also allow the determination of the cardiovascular cycle as a temporal quantity. 
[0111] The closeness between successive IVUS images is a parameter which clearly behaves in a periodic pattern. 
This is a result of the periodicity of most types of inter-image motion that are present. A closeness function may be 
formed in which each value results from a closeness operation between a pair of successive images. For example, a 
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set of ten images wifl produce nine successive closeness values. 

[0112] The closeness function can be derived from a cross-correlation type operation, SAD operation or any other 
type of operation that produces a closeness type of function. Normalized cross-correlation produces very good results 
when used for monitoring periodicity. 

[0113] The following formula shows the formula for the cross-correlation coefficient (as a function of the Nth image} 
for calculating the closeness function: 



Correlation Junction(N) = £B(.v. y) * A(.x,y) / V (E A(x,y)- * £ B(x,y) z ) 



where: 

'5 Correlation_function(N) = one dimensional function producing one value for every pair of images; 

A = predecessor image matrix (the Nth image); 
B = current image matrix (the Nth+1 image); 
* = multiplication of pixel by corresponding pixel; 
I = sum on all pixels in matrix. 

20 

[01141 The correlation coefficient is a byproduct of the stabilization process, because the central value (shiftX = 0, 
shiftY = 0) of the normalized cross-correlation matrix (C(shiftX, shiftY)) is always computed. This holds true for all types 
of closeness functions used for stabilization. The central value of the closeness matrix (C(shiftX = 0, shiftY = 0)), either 
cross-correlation or another type of operation used for stabilization, can always be used for producing a closeness 
25 function. 

[0115] The closeness function can also be computed from images which are shifted one in relation to another, i.e., 
Hie value used to form the function is C(shiftX, shiftY) where shiftX and shiftY are not equal to zero. The Closeness 
function need not necessarily be formed from whole images but can also be calculated from parts of images, either 
corresponding or shifted in relation to one another. 

30 [0116] Figure 9 shows an ECG and cross-correlation coefficient plotted graphically in synchronous fashion. Both 
curves are related to the same set of images. Figure 9(a) shows a graph of the ECG signal and Figure 9(b) shows a 
graph of the cross-correlation coefficient derived from successive IVUS images. The horizontal axis displays the image 
number (a total of 190 successive images). As can be observed, the cross-correlation coefficient function in Figure 9 
(b) shows a periodic pattern, and its periodicity is the same as that displayed by the ECG signal in Figure 9(a) {both 

36 show approximately six heart beats). 

[0117] Monitoring the periodicity of the closeness function may be complicated because the closeness function does 
not have a typical shape, it may vary in time, it depends on the type of closeness function used, and it may vary from 
vessel segment to vessel segment and from subject to subject. 

[0118] To monitor the periodicity of the closeness function continuously and automatically a variety of methods may 
^o be employed. One method, for example, is a threshold type method. This method monitors for a value of the closeness 

function over a certain value known as a threshold. Once this value is detected, the method monitors for when the 

threshold is again crossed. The period is determined as the difference in time between the crossings of the threshold. 

An example of this method is shown in Figure 1 0 as a table. The table shows a group of cross-correlation coefficient 

values (middle row) belonging to successive images (numbers 1 through 10 shown in the top row). If the threshold, 
45 for example, is set to the value of 0.885, then this threshold is first crossed in the passage from image #2 to image #3. 

The threshold is crossed a second time in the passage from image #6 to image #7. Thus, the time period of the 

periodicity is the time taken to acquire 7-3 = 4 images. 

[01 1 9] Another method that can be used to extract the cardiac periodicity from the closeness curve is internal cross- 
correlation. This method utilizes a segment of the closeness function, i.e., a group of successive values. For example, 

so in the table shown in Figure 10, the segment may be comprised of the first four successive images, i.e., images #1 
through #4. Once a segment is chosen, it is cross-correlated with itself, producing a cross-correlation value of 1 . Next, 
this segment is cross-correlated with a segment of the same size extracted from the closeness function, but shifted 
one image forward. This is repeated, with the segment shifted two images forward, and so on. In the example shown 
in Figure 10, the segment {0.8, 0.83, 0.89, 0.85} would be cross-correlated with a segment shifted by one image {0.83, 

55 0.89, 0.85, 0.82}, then the segment {0.8, 0.83, 0.89, 0.85} would be cross-correlated with a segment shifted by two 
images {0.89, 0.85, 0.82, 0.87}, and so on. The bottom row of the table in Figure 10 shows the results of these internal 
cross-correlations. The first value of 1 is a result of the cross-correlation of the segment with itself. These cross-cor- 
relation values are examined to determine the location of the local maxima. In this example, they are located in image 
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#1 and image #5 (their values are displayed in bold). The resulting periodicity is the difference between the location 
of the local maxima and the location from which the search was initiated {i.e., image #1). In this example, the periodicity 
is the time that elapsed from the acquisition of image # 1 to image #5, which is 5 - 1 = 4 images. Once a period has 
been detected, the search begins anew using a segment surrounding the local maximum, e.g., image #5. In this ex- 

5 ample, for example, the new segment could be the group of closeness values belonging to images #4 through #7. 
[0120] Due to the nature of the type of calculation involved, the internal cross-correlation operation at a certain point 
in time requires the closeness values of images acquired at a future time. Thus, unlike the threshold method, the 
closeness method requires the storage of images (in memory) and the periodicity detection is done retrospectively. 
The cardiac periodicity can also be monitored by transforming the closeness curve into the temporal frequency domain 

10 by the Fourier transform. In the frequency domain the periodicity should be expressed as a peak corresponding to the 
periodicity. This peak can be detected using spectral analysis. 

[0121] The closeness function can provide additional important information about IVUS images which cannot be 
extracted from external signals, such as ECG, that are not derived from the actual images. The behavior of this function 
can indicate certain states in the IVUS images or image parts used to form the closeness function. Important features 
15 in the closeness function which are indicative of the state of the IVUS images are the presence of periodicity and the 
"roughness" of the closeness function. Normal IVUS images should exhibit a relatively smooth and periodic closeness 
function as displayed, for example, in Figure 9(b). 

[0122] However, if "roughness" and/or periodicity are not present then this could indicate some problem in the for- 
mation of IVUS images, i.e., the presence of an artifact in the image formation caused by, for example, either a me- 

20 chanicai or electronic malfunction. The following figure helps to illustrate this. Figure 11 shows a graph of the cross- 
correlation coefficient derived from successive IVUS images. This graph is analogues, in its formation, to the cross- 
correlation plot in Figure 9(b), but in this example it is formed by a different imaging catheter used in a different subject. 
[0123] In this example, it is clear that the closeness function does not exhibit clear periodicity nor does it have a 
smooth appearance but rather a rough or spiky appearance. In this case the behavior of the closeness graph was 

25 caused by the non-uniformity of the rotation of the IVUS transducer responsible for emitting/collecting the ultrasonic 
signals displayed in the image. This type of artifact sometimes appears in IVUS catheter-transducer assemblies in 
which there are moving mechanical parts. 

[0124] The closeness function, when considered to reflect normal imaging conditions, can serve fora furtherpurpose. 
This is linked with the location of the maxima in each cycle of the closeness function. Locating these maxima may be 
so important for image processing algorithms which process several successive images together. Images found near 
maxima images tend to have high closeness and little inter-image motion, one in relation to the other. Additionally, if 
images belonging to the same phase of successive cardiac cycles are required to be selected, it is usually best to 
select them using the maxima (of the closeness function) in each cycle. 

[01 25] In one display method, for example, these images are projected onto the display and the gaps are filled in by 

35 interpolated images. By this display method all types of periodic motion can be stabilized. 

[0126] The shift logic stage in the stabilization process can also make use of cardiovascular periodicity monitoring. 
If drift is to be avoided, the accumulated shift after each (single) cardiac cycle should be small or zero, i.e., the sum of 
all shifts over a period of a cycle should result in zero or near zero. This means that the drift phenomena can be limited 
by utilizing shift logic which is coupled to the periodicity monitoring. 

io [0127] Referring now to Figure 12, most IVUS images can be divided into three basic parts. The central area (around 
the catheter), labeled as Lumen in Figure 12, is the actual lumen or interior passageway (cavity) through which fluid, 
e.g., blood flows. Around the lumen, is the actual vessel, labeled Vessel in Figure 12, composed of several layers of 
tissue and plaque (if diseased). Surrounding the vessel is other tissue, labeled Exterior in Figure 12, i.e., muscle or 
organ tissue, for example, the heart in the coronary vessel image. 

45 [0128] When IVUS images are viewed dynamically (i.e., In film format), the display of the interior, where the blood 
flows, and of the exterior surrounding the vessel, usually shows a different temporal behavior than the vessel itself. 
[0129] Automatically monitoring the temporal behavior of pixels in the dynamic IVUS image would allow use of the 
information extracted by the process to aid in interpretation of IVUS images. This information can be used to enhance 
IVUS displays by filtering and suppressing the appearance of fast changing features, such as fluid, e.g., blood, and 

50 the surrounding tissue, on account of their temporal behavior. This information can also be used for automatic seg- 
mentation, to determine the size of the lumen automatically by identifying the fluid, e.g., blood, and the surrounding 
tissue based on the temporal behavior of textural attributes formed by their composing pixels. 
[0130] To accomplish automatic monitoring of temporal behavior there must be an evaluation of the relationship 
between attributes formed by corresponding pixels belonging to successive images. Extraction of temporal behavior 

55 bears resemblance to the methods used for closeness operations on a localized basis, as described previously. 

[0131] High temporal changes are characterized by relatively large relative gray value changes of corresponding 
pixels, when passing from one image to the next. These fast temporal changes may be suppressed in the display by 
expressing these changes through the formation of a mask which multiplies the original image. This mask reflects 
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temporal changes in pixel values. A problem that arises in this evaluation is determining whether gray value changes 
in corresponding pixel values are due to either flow or change in matter, or movements of the vessel/catheter. By 
performing this evaluation on stabilized images overcomes or at least minimizes this problem. 
[0132] The following definitions apply: 



B = current (stabilized or non-stabilized) image; 

A = predecessor (stabilized or non-stabilized) image; 

C = successor (stabilized or non-stabifized) image; 



abs = absolute value. 

[0133] The matrices used can be either in Cartesian or Polar form, 

[0134] The following operation, resulting in a matrix D1, shall be defined as follows: D1 is a matrix, in which each 
pixel with coordinates X, Y is the sum of the absolute differences of its small surrounding neighborhood, e.g., 9 elements 
(X-2:X+2, Y-2:Y+2 - a 3 x 3 square), extracted from images A and B, respectively. 

[0135] For example, the following illustration shows corresponding pixels (in bold) and their close neighborhood in 
matrices A and B. 



6715 
3 5 83 



3 4 70 
2 1 6 



[0136] The pixel in matrix D1, with the location corresponding to the pixels with value 4 (in B) and 7 (in A) will be 
assigned the following value: 



abs(1-3) + abs (4-6) + abs(51-8) + abs(6-3) + 
30 abs(7-4) + abs(1 5-70) + abs(3-2) + abs(5-1 ) + abs(83-6) = 1 90 

[0137] D2 is defined similarly but for matrices B and C. 

[0138] D1 and D2 are, in effect, difference matrices which are averaged by using the 3 x 3 neighborhood in order to 
diminish local fluctuations or noise. Large gray value changes between images A and B or between B and C will be 
as expressed as relatively high values in matrices D1 and D2 respectively. 

101 39] Anew matrix, Dmax is nexl formed, in which every pixel is the maximum of the corresponding pixels in matrices 
D1 and D2: 



40 Dmax = max(D1, D2) 

where: 

max(D1, D2) = each pixel in Dmax holds the highest of the two corresponding pixels in D1 and D2. 
[0140] Thus, the single matrix Dmax particularly enhances large pixel changes between matrices A, B and C. A mask 
45 matrix (MD), is then formed from Dmax by normalization, i.e., each pixel in Dmax is divided by the maximal value of 
Dmax. Therefore, the pixel values of the mask MD range from zero to one. 

[0141] The role of the mask is to multiply the current image B in the following manner, forming a new matrix or image 
defined as BOUT: 



BOUT = (1-MD ) * B 



B = original current image; 
BOUT = the new image; 

n = each pixel in the matrix MD is raised to the power of n. n is generally a number with a value, for example, of 2-1 0; 
1-MD" = a matrix in which each pixel's value is one minus the value of the corresponding pixel in MD, 
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[0142] By performing the subtraction 1-MD n , small values of MD which reflect slow changing features become high 
values in 1-MD n . Moreover, the chance that only slow changing features will have high values is increased because 
of the prior enhancement of high MD values (by forming MD as a maximum between matrices D1 and D2). 
[0143] The multiplication of the mask (1-MD n ) by the current image B, forms a new image BOUT in which the ap- 
5 pearance of stow changing pixels are enhanced while fast changing pixels' values are decreased. The number n de- 
termines how strong the suppression of fast changing features will look on the display. 

[0144] Figure 13 illustrates the results of temporal filtering. The left image is an original IVUS image (i.e., matrix B) 
from a coronary vessel, as it would look on the current display. The right image has undergone the processing steps 
described above, i.e., temporal filtering (matrix BOUT). Note that in the right image, blood and the surrounding tissue 

10 is filtered (suppressed) and lumen and vessel borders are much easier to identify. 

[0145] Automatic segmentation differentiates fluid, e.g., blood and exterior, from the vessel wall based on the differ- 
ences between the temporal behavior of a textura! quality. As in the case of temporal filtering, this method is derived 
from the relationship between corresponding pixels from a number of successive images. If pixel values change be- 
cause of inter-image motion, then performance of the algorithm will be degraded. Performing stabilization prior to 

75 automatic segmentation will overcome, or at least minimize this problem. 

[0146] As in the case of temporal filtering, the following definitions shall apply: 



B = current (stabilized or non-stabilized) image; 

A = predecessor (stabilized or non-stabilized) image; 

C = successor (stabilized or non-stabiiized) image. 



[0147] The matrices can be either in Cartesian or Polar form. 

[0148] The textural quality can be defined as follows: Suppose the four nearest neighbors of a pixel with value "a" 
are "b," *c," "d" and "e," then the classification of "a" will depend on its relations with "b," "c," "d" and "e." This can be 
25 shown with the following illustration: 



[0149] The following categories can now be formed: 
35 [0150] In the vertical direction: 

if a>b and a>e then "a" is classified as belonging to the category I; 
if a>b and a<e then "a" is classified as belonging to the category II; 
if a<b and a<e then "a" is classified as belonging to the category 111; 
40 if a<b and a>e then "a" is classified as belonging to the category IV; 

if a=b or a=e then "a" is classified as belonging to the category V. 

[0151] In the horizontal direction: 

*s if a>c and a>d then "a" is classified as belonging to the category I; 

if a>c and a<d then "a" is classified as belonging to the category II; 
if a<c and a<d then "a" is classified as belonging to the category III; 
if a<c and a>d then "a" is classified as belonging to the category IV; 
if a=c or a=d then "a" is classified as belonging to the category V. 

50 

[0152] The vertical and horizontal categories are next combined to form a new category. As a result, pixel "a" can 
: now belong to 5 x 5 = 25 possible categories. This means that the textural quality of "a" is characterized by its belonging 
to one of those (25) categories. 
[0153] For example, in the following neighborhood: 

55 
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7 

10 10 14 

3 



[0154] Pixel "a" = 1 0 is classified as belonging to the category which includes category I vertical (because 10>7and 
10>3) and category V horizontal (because 10=10). However, if pixel "a" would have been situated in the following 
neighborhood: 



7 

11 10 14 

3 

it would have been classified as belonging to a different category because its horizontal category is now category III 
(10<11 and 10<14). 

[0155] By determining the relationship of each pixel to its close neighborhood a textural quality has been formed 
20 which classifies each pixel into 25 possible categories. The number of categories may vary (increased or decreased), 
i.e., for example, by changing the categorizing conditions, as may the number of close neighbors used, for example, 
instead of four, eight close neighbors may be used. 

[0156] The basic concept by which the textural changes are used to differentiate fluid, e.g., blood, from the vessel 
is by monitoring the change in categories of corresponding pixels in successive images. To accomplish this the category 
25 in each and every pixel in matrices A, B and C are determined. Next, corresponding pixels are each tested to see if 
this category has changed. If it has, the pixel is suspected of being a fluid, e.g., blood, or surrounding tissue pixel. If it 
has not changed, then the pixel is suspected of being a vessel pixel. 

[0157] The following example shows three corresponding pixels (with values 8, 12 and 14) and their neighborhoods 
in successive matrices A, B and C. 

30 



9 

19 12 1 
100 



21 14 17 
20 



[0158] In this example, the category of the pixel valued 12 (in B) is the same as in A and C, so it will be classified as 
40 a pixel with a higher chance of being a vessel wall pixel. If, however, the situation was as shown below (20 in C changes 
to 13): 



9 

19 12 13 
100 



21 14 17 

13 



then pixels 8 in A and 12 in B have the same categories, but 14 in C has a different category as in the prior example. 
As a result, pixel 12 in B will be classified as a pixel with a higher chance of being a fluid (lumen), i.e., blood, or exterior 
tissue pixel. 

[0159] The classification method described so far monitors the change in the texture or pattern associated with the 
55 small neighborhood around each pixel. Once this change is determined as described above, each pixel can be assigned 
a binary value. For example, a value of 0, if it is suspected to be a vessel pixel, or a value of 1 , if it is suspected to be 
a blood pixel or a pixel belonging to the vessel's exterior. The binary image, serves as an input for the process of 
identification of the lumen and the original pixel values cease to play a role in the segmentation process. 
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[0160] Identification of the lumen using the binary image is based on two assumptions which are generally valid in 
IVUS images processed in the manner described above. The first, is that the areas in the image which contain blood 
or are found on the exterior of the vessel are characterized by a high density of pixels with a binary value of 1 (or a 
low density of pixels with a value of zero). The term density is needed because there are always pixels which are 
5 misclassified. The second assumption, is that from a morphological point of view, connected areas of high density of 
pixels with the value of 1 (lumen) should be found around the catheter and surrounded by connected areas of low 
density of pixels with the value of 1 (vessel) which are in turn, surrounded again by connected areas of high density 
of pixeis with the value of 1 (vessel's exterior). The reason for this assumption is the typical morphological arrangement 
expected from a blood vessel. 

10 [01 61 ] These two assumptions form the basis of the subsequent processing algorithm which extracts the actual area 
associated with the lumen out of the binary image. This algorithm can utilize known image processing techniques, such 
as thresholding the density feature in localized regions (to distinguish blood/exterior from vessel ) and morphological 
operators such as dilation or linking to inter-connect and form a connected region which should represent the actual 
lumen found within the vessel wall limits. 

is [0162] Figure 14 shows an image of the results of the algorithm for automatic extraction of the lumen. The image is 
an original IVUS image (for example, as described above as image B) and the lumen borders are superimposed (by 
the algorithm) as a bright line. The algorithm for the extraction of the lumen borders was based on the monitoring of 
the change in the texturaf quality described above, using three successive images. 

[0163] The examples described above of temporal filtering and automatic segmentation include the use of two ad- 
20 ditional images (for example, as described above as images A and C) in addition to the current image (for example, 
as described above as image B). However, both of these methods could be modified to utilize less {i.e., only one 
additional image) or more additional images. 

[0164] The performance of the two methods described above will be greatly enhanced if combined with cardiovas- 
cular periodicity monitoring. This applies, in particular, to successive images in which cardiovascular periodicity mon- 
25 itoring produces high inter-image closeness values. Those images usually have no inter-image motion. Thus, most 
reliable results can be expected when successive images with maximal inter-image closeness are fed as inputs to 
either temporal filtering or automatic segmentation. 

[0165] During treatment of vessels using catheterization, it is a common practice to repeat IVUS pullback examina- 
tions in the same vessel segment. For example, a typical situation is first to review the segment in question, evaluate 

so the disease (if any), remove the IVUS catheter, consider therapy options, perform therapy and then immediately after 
(during the same session) examine the treated segment again using IVUS in order to assess the results of therapy. 
[0166] To properly assess the results of such therapy, corresponding segments of the pre-treatment and post-treat- 
ment segments which lie on the same locations along the length of the vessel, I.e., corresponding segments, should 
be compared. The following method provides for matching, i.e., automatic identification (registration) of corresponding 

35 segments. 

[0167] To accomplish matching of corresponding segments, closeness/similarity operations are applied between 
images belonging to a first group of successive images, i.e., a reference segment, of a first pullback film and images 
belonging to a second group of successive images of a second pullback film. Matching of the reference segment in 
the first film to its corresponding segment in the second film is obtained when some criteria function is maximized. 

40 [0168] From either one of the two films a reference segment is chosen. The reference segment may be a group of 
successive images representing, for example, a few seconds of film of an IVUS image. It is important to select the 
reference segment from a location in a vessel which is present in the two films and has undergone no change as a 
result of any procedure, i.e., the reference segment is proximal or distal to the treated segment. 
[0169] As an example, the table in Figure 15 will help clarify the method for matching of corresponding segments. 

45 [0170] The left column shows the time sequence of the first film, in this case the film consists of twenty successive 
images. The middle column, shows the reference segment which is selected from the second film and consists of 10 
successive images. The right column lists the 10 successive images from the first film (#5 - #14) which actually corre- 
spond to (or match) the images of the reference segment from the second film (#1 - #1 0). The purpose of the matching 
process is to actually reveal this correspondence. 

so [0171] Once a reference segment is chosen, it is shifted along the other film, one image (or more) each time, and a 
set of stabilization and closeness operations are performed between the corresponding images in each segment. The 
direction of the shift depends on the relative location of the reference segment in the time sequence of the two films. 
However, in general, if this is not known, the shift can be performed in both directions. 
[0172] For example, where: 

55 

r = reference segment; and 
f = first film, 
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the first set of operations will take place between the images comprising the following pairs: r#1-f#1, r#2-f#2, 
r#3-f#3 r#10-f#10. 

[0173] The second set of operations will take place between the images comprising the following pairs: r#1-f#2, 
r#2-f#3, r#3-f#4 ...... r#10-f#11. 

s [0174] The third set of operations will take place between the images comprising the following pairs: r#1 -f#3, r#2-f#4, 
r#3-f#5, . . . ,f#10-f#12,and soon, etc. As can be observed in this example, the shifting is performed, by a single image 
each time and in one direction only. 

[0175] For example, the following operations between the images in each pair may be performed. First, an image 
from the reference segment is stabilized for rotational and Cartesian motion, in relation to its counterpart in the first 

io film. Then closeness operations are performed between the images in each pair. This operation can be, for example, 
normalized cross-correlation (discussed above in relation to periodicity detection). Each such operation produces a 
' closeness value, for example, a cross-correlation coefficient when normalized cross-correlation is used. A set of such 
operations will produce a number of cross-correlation values. In the example shown in the table of figure 15, each time 
the reference segment is shifted, ten new cross-correlation coefficients will be produced. 

is [0176] The closeness values produced by a set of operations can then be mapped into some type of closeness 
function, for example, an average function. Using the above example, the cross-correlation coefficients are summed 
up and then divided by the number of pairs, i.e., ten. Each set of operations results therefore, in a single value, i.e., 
an average closeness, which should represent the degree of closeness between the reference segment and its tem- 
porary counterpart in the first film. Thus, the result of the first set of operations will be a single value, the result of the 

20 second set of operations will be another value, etc. 

[0177] We can expect that the maximal average closeness will occur as a result of the operations performed between 
segments which are very alike, i.e., corresponding or matching segments. 

[0178] In the above example, these segments should be matched during the fifth set of operations which take place 
between the images comprising the following pairs: r#1-f#5, r#2-f#6, r#3-f#7 r#10-f#14. 

25 [0179] The maximal average closeness should, therefore, indicate corresponding segments because each pair of 
images are, in fact, corresponding images, i.e., they show the same morphology. The criteria might not, however, follow 
this algorithm. It may, for example, take into account the form of the closeness function, derived from many shifted 
segment positions instead of using only one of its values which turns out to be the maximum. 
[0180] Once corresponding segments are identified, the complete first and second films may be synchronized one 

30 jn relation to the other, This will be a result of an appropriate frame shift, revealed by the matching process, implemented 
in one film in relation to the other. Thus, when watching the two films side by side, the pre-treated segment will appear 
concurrently with the post-treated section. 

[0181] Besides for synchronizing the corresponding segments, the above operation also stabilizes the corresponding 
segments one in relation to the other. This further enhances the ability to understand the changes in morphology. Thus, 

35 even though when the catheter is reinserted in the vessel its position and orientation are likely to have changed, nev- 
ertheless, the images in the pre-treatment and post-treatment films will be stabilized in relation to each other. 
[0182] The number of images used for the reference segment may vary. The more images used in the matching 
process, the more robust and less prone to local errors it will be. However, the tradeoff is more computational time 
required for the calculations for each matching process as the number of pairs increases. 

40 [0183] It is important in acquiring the pullback films that the pullback rate remains stable and is known. It is preferred 
that the pullback rate be identical in the two acquisitions. 

[0184] Many different variations of the present invention are possible. The various features described above may be 
incorporated individually and independently of one another. These features may also be combined in various groupings. 

45 

Claims 

1. An apparatus for performing an intravascular ultrasound imaging via an ultrasound signal transmitter and detector 
capable of being placed within a bodily lumen, characterized by: 

50 

an arrangement for detecting ultrasound signals; 

an arrangement for deriving a first image from a first set of detected ultrasound signals; 
an arrangement for processing and digitizing the first set of detected ultrasound signals; 
an arrangement for deriving a first two-dimensional array from the digitized first set of detected ultrasound 
55 signals, the first two-dimensional array including a first plurality of elements wherein the first image is configured 

as the first two-dimensional array; 

an arrangement for deriving a second image from a second set of the detected ultrasound signals; 
an arrangement for processing and digitizing the second set of detected ultrasound signals; 
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an arrangement for deriving a second two-dimensional array from the digitized second set of detected ultra- 
sound signals, wherein the second image is configured as the second two-dimensional array, the second two- 
dimensional array including a second plurality of elements, each one of the first plurality of elements and the 
second plurality of elements representing a detected ultrasound signal from a predetermined spatial location; 
and 

an arrangement for performing a shift evaluation of the second image in relation to the first image in order to 
detect a motion. 

2. The apparatus according to claim 1 , wherein: 

each one of the first two-dimensional array and the second two-dimensional array is configured in polar coor- 
dinates, and 

the shift evaluation is performed in polar coordinates along at least one dimension. 

3. The apparatus according to claim 2, wherein the detected motion corresponds to at least one of a rotational move- 
ment and a vasomotion. 

4. The apparatus according to claim 3, wherein: 

the rotational movement corresponds to at least one of a global rotational movement, a rigid rotational move- 
ment, and a local rotational movement, and 

the vasomotion corresponds to at least one of a global vasomotion and a local vasomotion. 

5. The apparatus according to claim 3, further comprising an arrangement for stabilizing the second image in relation 
to the first image for at least one of the rotational movement and the vasomotion, wherein the stabilizing performed 
by the arrangement for stabilizing is performed in polar coordinates. 

6. The apparatus according to claim 5, wherein the arrangement for stabilizing stabilizes at least one of a global 
rotation, a rigid rotation, a local rotation, a local vasomotion, and a global vasomotion. 

7. The apparatus according to claim 6, wherein the arrangement for stabilizing, at least one of the global rotation and 
the rigid rotation and the global vasomotion includes an arrangement for shifting the second image in polar coor- 
dinates along at least one dimension according to a magnitude and a direction derived from a result of the shift 
evaluation. 

8. The apparatus according to claim 7, further comprising an arrangement for limiting a drift occurring due to the 
stabilizing of the second image in relation to the first image. 

9. The apparatus according to claim 8, wherein the arrangement for limiting the drift includes an arrangement for 
shifting the second image by a magnitude that is adjusted by using information derived from a cardiovascular 
periodicity monitoring. 

10. A method for performing an intravascular ultrasound imaging via an ultrasound signal transmitter and detector 
capable of being placed within a bodily lumen, characterized by: 

detecting ultrasound signals; 

deriving a first image from a first set of detected ultrasound signals; 
processing and digitizing the first set of detected ultrasound signals; 

deriving a first two-dimensional array from the digitized first set of detected ultrasound signals, the first two- 
dimensional array including a first plurality of elements wherein the first image is configured as the first two- 
dimensional array; 

deriving a second image from a second set of the detected ultrasound signals; 
processing and digitizing the second set of detected ultrasound signals; 

deriving a second two-dimensional array from the digitized second set of detected ultrasound signals, wherein 
the second image is configured as the second two-dimensional array, the second two-dimensional array in- 
cluding a second plurality of elements, each one of the first plurality of elements and the second plurality of 
elements representing a detected ultrasound signal from a predetermined spatial location; and 
performing a shift evaluation of the second image in relation to the first image in order to detect a motion. 



21 



EP 1 227 342 A1 



11. The method. according to claim 10, wherein: 

each one of the first two-dimensional array and the second two-dimensional array is configured in polar coor- 
dinates, and 

the shift evaluation is performed in polar coordinates along at least one dimension. 

12. The method according to claim 11, wherein the detected motion corresponds to at least one of a rotational move- 
ment and a vasomotion. 

13. The method according to 12, wherein: 

the rotational movement corresponds to at least one of a global rotational movement, a rigid rotational move- 
ment, and a local rotational movement, and 

the vasomotion corresponds to at least one of a global vasomotion and a local vasomotion. 

14. The method according to claim 12, characterized by stabilizing the second image in relation to the first image for 
at least one of the rotational movement and the vasomotion, wherein the stabilizing performed is performed in 
polar coordinates. 

15. The method according to claim 14, wherein the stabilizing stabilizes at least one of a global rotation, a rigid rotation, 
a local rotation, a local vasomotion, and a global vasomotion. 

16. The method according to claim 1 5, wherein the stabilizing, at least one of the global rotation and the rigid rotation 
and the global vasomotion includes shifting the second image in polar coordinates along at least one dimension 
according to a magnitude and a direction derived from a result of the shift evaluation. 

17. The method according to claim 1 6, characterized by limiting a drift occurring due to the stabilizing of the second 
image in relation to the first image. 

18. The method according to claim 17, wherein the limiting the drift includes shifting the second image by a magnitude 
that is adjusted by using information derived from a cardiovascular periodicity monitoring. 
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Figure 3 
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Figure 6 
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Figure 7 
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Figure 9 
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Figure 1 1 
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Figure 12 
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